Non-contact Measurement of Pulse Rate Variability Using a Webcam and Application to Mental Illness Screening System.

The COVID-19 pandemic is a global health crisis. Mental health is critical in such uncertain situations, particularly when people are required to significantly restrict their movements and change their lifestyles. Under these conditions, many countries have turned to telemedicine to strengthen and expand mental health services. Our research group previously developed a mental illness screening system based on heart rate variability (HRV) analysis, enabling an objective and easy mental health self-check. This screening system cannot be used for telemedicine because it uses electrocardiography (ECG) and contact photoplethysmography (PPG), that are not widely available outside of a clinical setting. The purpose of this study is to enable the extension of the aforementioned system to telemedicine by the application of non-contact PPG using an RGB webcam, also called imaging- photoplethysmography (iPPG). The iPPG measurement errors occur due to changes in the relative position between the camera and the target, and due to changes in light. Conventionally, in image processing, the pixel value of the entire face region is used. We propose skin pixel extraction to eliminate blinks, eye movements, and changes in light and shadow. In signal processing, the green channel signal is conventionally used as a pulse wave owing to the absorption characteristics of blood flow. Taking advantage of the fact that the red and blue channels contain noise, we propose a signal reconstruction method for removing noise and strengthening the signal in the pulse rate variability (PRV) frequency band by weighting the three signals of the RGB camera. We conducted an experiment with 13 healthy subjects, and showed that the PRV index and pulse rate (PR) errors estimated by the proposed method were smaller than those of the conventional method. The correlation coefficients between estimated values by the proposed method and reference values of LF, HF, and PR were 0.86, 0.69, and 0.96, respectively.

Contactless Vital Signs Measurement System Using RGB-Thermal Image Sensors and Its Clinical Screening Test on Patients with Seasonal Influenza.

Background: In the last two decades, infrared thermography (IRT) has been applied in quarantine stations for the screening of patients with suspected infectious disease. However, the fever-based screening procedure employing IRT suffers from low sensitivity, because monitoring body temperature alone is insufficient for detecting infected patients. To overcome the drawbacks of fever-based screening, this study aims to develop and evaluate a multiple vital sign (i.e., body temperature, heart rate and respiration rate) measurement system using RGB-thermal image sensors. Methods: The RGB camera measures blood volume pulse (BVP) through variations in the light absorption from human facial areas. IRT is used to estimate the respiration rate by measuring the change in temperature near the nostrils or mouth accompanying respiration. To enable a stable and reliable system, the following image and signal processing methods were proposed and implemented: (1) an RGB-thermal image fusion approach to achieve highly reliable facial region-of-interest tracking, (2) a heart rate estimation method including a tapered window for reducing noise caused by the face tracker, reconstruction of a BVP signal with three RGB channels to optimize a linear function, thereby improving the signal-to-noise ratio and multiple signal classification (MUSIC) algorithm for estimating the pseudo-spectrum from limited time-domain BVP signals within 15 s and (3) a respiration rate estimation method implementing nasal or oral breathing signal selection based on signal quality index for stable measurement and MUSIC algorithm for rapid measurement. We tested the system on 22 healthy subjects and 28 patients with seasonal influenza, using the support vector machine (SVM) classification method. Results: The body temperature, heart rate and respiration rate measured in a non-contact manner were highly similarity to those measured via contact-type reference devices (i.e., thermometer, ECG and respiration belt), with Pearson correlation coefficients of 0.71, 0.87 and 0.87, respectively. Moreover, the optimized SVM model with three vital signs yielded sensitivity and specificity values of 85.7% and 90.1%, respectively. Conclusion: For contactless vital sign measurement, the system achieved a performance similar to that of the reference devices. The multiple vital sign-based screening achieved higher sensitivity than fever-based screening. Thus, this system represents a promising alternative for further quarantine procedures to prevent the spread of infectious diseases.